Search for: All records

Language model (LM) agents are increasingly used as autonomous decision-makers which need to actively gather information to guide their decisions. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world—key to robust, scientifically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclusions. In this work, we examine LMs’ ability to explore and infer causal relationships, using the well-established Blicket Test paradigm from developmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the unusual, yet equally (or sometimes even more) evidenced conjunctive ones. This “disjunctive bias” persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their training data. To this end, we quantify similarities between LMs and humans, finding that LMs exhibit adult-like inference profiles (but not child-like). Finally, we propose a test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning. 

In the face of difficult exploration problems in reinforcement learning, we study whether giving an agent an object-centric mapping (describing a set of items and their attributes) allow for more efficient learning. We found this problem is best solved hierarchically by modelling items at a higher level of state abstraction to pixels, and attribute change at a higher level of temporal abstraction to primitive actions. This abstraction simplifies the transition dynamic by making specific future states easier to predict. We make use of this to propose a fully model-based algorithm that learns a discriminative world model, plans to explore efficiently with only a count-based intrinsic reward, and can subsequently plan to reach any discovered (abstract) states. We demonstrate the model's ability to (i) efficiently solve single tasks, (ii) transfer zero-shot and few-shot across item types and environments, and (iii) plan across long horizons. Across a suite of 2D crafting and MiniHack environments, we empirically show our model significantly out-performs state-of-the-art low-level methods (without abstraction), as well as performant model-free and model-based methods using the same abstraction. Finally, we show how to learn low level object-perturbing policies via reinforcement learning, and the object mapping itself by supervised learning. 

We propose using surface and aerial shared autonomous electric vehicles (SAEVs) to improve the resilience of infrastructure and communities, or SAEV-R. In disruptive events, SAEVs can be temporarily deployed to evacuate and rescue at-risk populations, provide essential supplies and services to vulnerable households, and transport repair crews and equipment. We present a modeling framework for feasibility analysis and strategic planning associated with deploying SAEVs for disaster relief. The framework guides our examination of three scenarios: a hurricane-induced power outage, a pandemic-affected vulnerable population, and earthquake-damaged infrastructure. The results demonstrate the flexibility of the proposed framework and showcase the potential and versatility of SAEV-R systems to improve resilience. 

In recent years, aiming to enhance and extend user experiences beyond the real world, Extended Reality (XR) has emerged to become a new paradigm that enables a plethora of applications [1], e.g., online gaming, online conferencing, social media, etc. XR refers to the human-machine interactions that combine real and virtual environments with the support of computing/communications technologies and wearable devices. The XR content is generated by providers or other users, including audio, video and other metadata. In general, the generated XR content is transmitted to XR devices and rendered into XR scenes (i.e., to generate an image from a 2D or 3D model by means of a computer program), where users can experience a hybrid experience of the real and virtual worlds.